---
title: 'Get started with Kafka'
sidebarTitle: 'Kafka'
description: 'Learn how to setup Postgres change data capture (CDC) to Kafka in under 10 minutes. Setup a robust pipeline with backfills, inserts, updates, and deletes.'
---

import QuickstartInitialSteps from '/snippets/quickstart-initial-steps.mdx';
import QuickstartSourceStep from '/snippets/quickstart-source-step.mdx';
import QuickstartBackfillStep from '/snippets/quickstart-backfill-step.mdx';

In this quickstart, you'll create a real-time data pipeline that streams changes from a Postgres database to a Kafka topic. You'll:

- Boot Sequin
- Connect to a sample playground database
- Configure a Kafka topic to receive database changes
- See your changes flow in real-time

By the end, you'll have hands-on experience setting up Postgres change data capture (CDC) with Sequin and Kafka.

<Tip>
  This is the quickstart for streaming Postgres to Kafka. See the [how-to guide](/how-to/stream-postgres-to-kafka) for an explanation of how to use the Kafka sink or the [reference](/reference/sinks/kafka) for details on all configuration options.
</Tip>

<Steps titleSize="h2">
 <Step title="Boot Kafka (optional)" icon="database">
  If you don't already have Kafka running, start Kafka with Docker Compose:

  1. Download our <a href="https://github.com/sequinstream/sequin/raw/main/docs/static/kafka-docker-compose.yaml" download="docker-compose.yaml">Docker compose file for Kafka</a> (right click, save link as...).
  2. Move it to a new directory, navigate to that directory, and start the services:

  ```bash
  mkdir kafka-quickstart && cd kafka-quickstart
  # On macOS:
  mv ~/Downloads/kafka-docker-compose.yaml docker-compose.yaml
  docker compose up
  ```

  <AccordionGroup>
    <Accordion title="Alternative: Download with curl">
      ```bash
      mkdir kafka-quickstart && cd kafka-quickstart
      curl -o docker-compose.yaml https://raw.githubusercontent.com/sequinstream/sequin/main/docs/static/kafka-docker-compose.yaml
      docker compose up
      ```
    </Accordion>

    <Accordion title="Alternative: Copy/paste raw Kafka docker-compose.yaml">
      ```yaml
      name: playground-kafka
      version: '3'

      services:
        zookeeper:
          image: confluentinc/cp-zookeeper:latest
          environment:
            ZOOKEEPER_CLIENT_PORT: 2181
            ZOOKEEPER_TICK_TIME: 2000
          ports:
            - "2181:2181"

        kafka:
          image: confluentinc/cp-kafka:latest
          depends_on:
            - zookeeper
          ports:
            - "9092:9092"
          environment:
            KAFKA_BROKER_ID: 1
            KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
            KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://host.docker.internal:9092
            KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
            KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      ```
    </Accordion>
  </AccordionGroup>
  <Note>
    If you're using another Kafka instance, ensure you have the connection details ready.
  </Note>
 </Step>

 <Step title="Create the Kafka topic" icon="plus">
  Next, create the Kafka topic that will receive our database changes. Create this using the Kafka container you just started:

  ```bash
  docker exec playground-kafka-kafka-1 \
    kafka-topics --create \
    --topic products \
    --bootstrap-server localhost:9092 \
    --partitions 1 \
    --replication-factor 1
  ```

  You should see output confirming the topic was created:
  ```plaintext
  Created topic products.
  ```
 </Step>

 <QuickstartInitialSteps />
 <Step title="Create a Kafka Sink" icon="database">
  With the playground database connected, you can create a [sink](/reference/sinks/overview). This sink will send changes to the `products` table to your Kafka topic:

  <Steps>
    <Step title="Navigate to Sinks">
      Click "Sinks" in the sidebar navigation, then click "Create Sink".
    </Step>

    <Step title="Select sink type">
      Select "Kafka" as the sink type and click "Continue".
    </Step>

    <QuickstartSourceStep />
    <QuickstartBackfillStep />

    <Step title='Configure "Kafka Configuration"'>
      In the "Kafka Configuration" card, enter your Kafka connection details:

      - Hosts: If running locally with the provided docker-compose, use `host.docker.internal:9092`
      - Topic: The Kafka topic to stream to (e.g., `products`)
      - SASL Mechanism: Select if your Kafka cluster requires authentication
      - Username/Password: Required if SASL is enabled
      - TLS: Toggle on if your Kafka cluster requires TLS

      <Frame>
        <img style={{ maxWidth: '700px' }} src="/images/quickstart/kafka/kafka-config-card.png" alt="Kafka configuration card" />
      </Frame>
    </Step>

    <Step title="Test the connection">
      At the bottom of the form, click the "Test Connection" button. If you provided proper credentials, it should succeed.

      <Check>
        Sequin can connect to your Kafka cluster.
      </Check>
    </Step>

    <Step title="Create the sink">
      You can leave the rest of the defaults. As configured, the Kafka topic will first receive a backfill of all rows currently in the `products` table. Then, it will receive all changes to the `products` table in real-time.

      Click "Create Sink" to finish setting up your Kafka sink.
    </Step>
  </Steps>
 </Step>

  <Step title="See changes flow to your Kafka topic" icon="waveform-lines">
    On the new sink's overview page, you should see the "Health" status turn green, indicating data is flowing to your Kafka topic.

    Let's confirm messages are flowing:

    <Steps>
    <Step title="Messages tab">
      Click the "Messages" tab. You'll see a list of the recently delivered messages:

      <Frame>
        <img style={{ maxWidth: '700px' }} src="/images/quickstart/kafka/messages-tab.png" alt="List of recently delivered messages" />
      </Frame>

      <Check>
        Sequin indicates it backfilled the `products` table to your Kafka topic.
      </Check>
    </Step>
      <Step title="View in Kafka CLI">
        In your terminal, run the following command to read from the topic:

        ```bash
        docker exec playground-kafka-kafka-1 \
          kafka-console-consumer --bootstrap-server localhost:9092 \
          --topic products \
          --from-beginning \
          --max-messages 10
        ```

        You should see the messages that were sent from Sequin. These are [`read` events](/reference/messages) from the initial backfill of the `products` table.

        <Check>
          Messages are flowing from Sequin to your Kafka topic.
        </Check>
      </Step>

      <Step title="Make some changes">
        Let's make some changes to the `products` table and see them flow to your topic.

        In your terminal, run the following command to insert a new row into the `products` table:

        ```bash
        docker exec -i sequin-sequin_postgres-1 \
          psql -U postgres -d sequin_playground -c \
          "insert into products (name, price) values ('Organic Honey (16 oz)', 12.99);"
        ```

        In another terminal, consume messages from your Kafka topic:

        ```bash
        docker exec playground-kafka-kafka-1 \
          kafka-console-consumer --bootstrap-server localhost:9092 \
          --topic products \
          --from-beginning
        ```

        You should see a message corresponding to the inserted row.

        Feel free to try other changes:

        <AccordionGroup>
          <Accordion title="Update a product's price">
            ```bash
            docker exec -i sequin-sequin_postgres-1 \
              psql -U postgres -d sequin_playground -c \
              "update products set price = 7.99 where name = 'Avocados (3 pack)';"
            ```
          </Accordion>

          <Accordion title="Change a product's name">
            ```bash
            docker exec -i sequin-sequin_postgres-1 \
              psql -U postgres -d sequin_playground -c \
              "update products set name = 'Organic Avocados (3 pack)' where name = 'Avocados (3 pack)';"
            ```
          </Accordion>

          <Accordion title="Delete a product">
            ```bash
            docker exec -i sequin-sequin_postgres-1 \
              psql -U postgres -d sequin_playground -c \
              "delete from products where name = 'Blueberries (6 oz)';"
            ```
          </Accordion>
        </AccordionGroup>

        Each change will appear in your Kafka topic within a few seconds.
      </Step>
    </Steps>
  </Step>
</Steps>

<Check>
  Great work!
</Check>

You've successfully:

- Set up a complete Postgres change data capture pipeline
- Loaded existing data through a backfill
- Made changes to the `products` table
- Verified changes are flowing to your Kafka topic

## Ready to stream

Now you're ready to connect your own database to Sequin and start streaming changes:

<CardGroup cols={2}>
  <Card title="Guide: Connect Postgres" icon="elephant" href="/connect-postgres">
    Connect your Postgres database to Sequin.
  </Card>
  <Card title="Guide: Setting up a Kafka sink" icon="database" href="/how-to/stream-postgres-to-kafka">
    Setup a Kafka sink to send changes to your topic.
  </Card>
</CardGroup>